131 research outputs found
Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution
Color-guided depth super-resolution (DSR) is an encouraging paradigm that
enhances a low-resolution (LR) depth map guided by an extra high-resolution
(HR) RGB image from the same scene. Existing methods usually use interpolation
to upscale the depth maps before feeding them into the network and transfer the
high-frequency information extracted from HR RGB images to guide the
reconstruction of depth maps. However, the extracted high-frequency information
usually contains textures that are not present in depth maps in the existence
of the cross-modality gap, and the noises would be further aggravated by
interpolation due to the resolution gap between the RGB and depth images. To
tackle these challenges, we propose a novel Symmetric Uncertainty-aware Feature
Transmission (SUFT) for color-guided DSR. (1) For the resolution gap, SUFT
builds an iterative up-and-down sampling pipeline, which makes depth features
and RGB features spatially consistent while suppressing noise amplification and
blurring by replacing common interpolated pre-upsampling. (2) For the
cross-modality gap, we propose a novel Symmetric Uncertainty scheme to remove
parts of RGB information harmful to the recovery of HR depth maps. Extensive
experiments on benchmark datasets and challenging real-world settings suggest
that our method achieves superior performance compared to state-of-the-art
methods. Our code and models are available at
https://github.com/ShiWuxuan/SUFT.Comment: 10 pages, 9 figures, accepted by the 30th ACM International
Conference on Multimedia (ACM MM 22
Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning
Federated learning is an important privacy-preserving multi-party learning
paradigm, involving collaborative learning with others and local updating on
private data. Model heterogeneity and catastrophic forgetting are two crucial
challenges, which greatly limit the applicability and generalizability. This
paper presents a novel FCCL+, federated correlation and similarity learning
with non-target distillation, facilitating the both intra-domain
discriminability and inter-domain generalization. For heterogeneity issue, we
leverage irrelevant unlabeled public data for communication between the
heterogeneous participants. We construct cross-correlation matrix and align
instance similarity distribution on both logits and feature levels, which
effectively overcomes the communication barrier and improves the generalizable
ability. For catastrophic forgetting in local updating stage, FCCL+ introduces
Federated Non Target Distillation, which retains inter-domain knowledge while
avoiding the optimization conflict issue, fulling distilling privileged
inter-domain information through depicting posterior classes relation.
Considering that there is no standard benchmark for evaluating existing
heterogeneous federated learning under the same setting, we present a
comprehensive benchmark with extensive representative methods under four domain
shift scenarios, supporting both heterogeneous and homogeneous federated
settings. Empirical results demonstrate the superiority of our method and the
efficiency of modules on various scenarios
An Empirical Study of CLIP for Text-based Person Search
Text-based Person Search (TBPS) aims to retrieve the person images using
natural language descriptions. Recently, Contrastive Language Image Pretraining
(CLIP), a universal large cross-modal vision-language pre-training model, has
remarkably performed over various cross-modal downstream tasks due to its
powerful cross-modal semantic learning capacity. TPBS, as a fine-grained
cross-modal retrieval task, is also facing the rise of research on the
CLIP-based TBPS. In order to explore the potential of the visual-language
pre-training model for downstream TBPS tasks, this paper makes the first
attempt to conduct a comprehensive empirical study of CLIP for TBPS and thus
contribute a straightforward, incremental, yet strong TBPS-CLIP baseline to the
TBPS community. We revisit critical design considerations under CLIP, including
data augmentation and loss function. The model, with the aforementioned designs
and practical training tricks, can attain satisfactory performance without any
sophisticated modules. Also, we conduct the probing experiments of TBPS-CLIP in
model generalization and model compression, demonstrating the effectiveness of
TBPS-CLIP from various aspects. This work is expected to provide empirical
insights and highlight future CLIP-based TBPS research.Comment: 13 pages, 5 fiugres and 17 tables. Code is available at
https://github.com/Flame-Chasers/TBPS-CLI
Self-delivery of N-hydroxylethyl peptide assemblies to the cytosol inducing endoplasmic reticulum dilation in cancer cells
Inspired by clinical studies on alcohol abuse induced endoplasmic reticulum disruption, we designed a N-hydroxylethyl peptide assembly to regulate the ER stress response in cancer cells. Upon coupling with a coumarin derivative via an ester linkage, a prodrug was synthesized to promote esterase-facilitated self-delivery of N-hydroxylethyl peptide assemblies around the ER, inducing ER dilation. Following this, ER-specific apoptosis was effectively and efficiently activated in various types of cancer cells including drug resistant and metastatic ones
Heterogeneous Federated Learning: State-of-the-art and Research Challenges
Federated learning (FL) has drawn increasing attention owing to its potential
use in large-scale industrial applications. Existing federated learning works
mainly focus on model homogeneous settings. However, practical federated
learning typically faces the heterogeneity of data distributions, model
architectures, network environments, and hardware devices among participant
clients. Heterogeneous Federated Learning (HFL) is much more challenging, and
corresponding solutions are diverse and complex. Therefore, a systematic survey
on this topic about the research challenges and state-of-the-art is essential.
In this survey, we firstly summarize the various research challenges in HFL
from five aspects: statistical heterogeneity, model heterogeneity,
communication heterogeneity, device heterogeneity, and additional challenges.
In addition, recent advances in HFL are reviewed and a new taxonomy of existing
HFL methods is proposed with an in-depth analysis of their pros and cons. We
classify existing methods from three different levels according to the HFL
procedure: data-level, model-level, and server-level. Finally, several critical
and promising future research directions in HFL are discussed, which may
facilitate further developments in this field. A periodically updated
collection on HFL is available at https://github.com/marswhu/HFL_Survey.Comment: 42 pages, 11 figures, and 4 table
Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark
Federated learning has emerged as a promising paradigm for privacy-preserving
collaboration among different parties. Recently, with the popularity of
federated learning, an influx of approaches have delivered towards different
realistic challenges. In this survey, we provide a systematic overview of the
important and recent developments of research on federated learning. Firstly,
we introduce the study history and terminology definition of this area. Then,
we comprehensively review three basic lines of research: generalization,
robustness, and fairness, by introducing their respective background concepts,
task settings, and main challenges. We also offer a detailed overview of
representative literature on both methods and datasets. We further benchmark
the reviewed methods on several well-known datasets. Finally, we point out
several open issues in this field and suggest opportunities for further
research. We also provide a public website to continuously track developments
in this fast advancing field: https://github.com/WenkeHuang/MarsFL.Comment: 22 pages, 4 figure
Enzyme-mediated dual-targeted-assembly realizes a synergistic anticancer effect
We designed and synthesized homochiral-peptide-based boron diketonate complexes. Co-administration of the two stereoisomers in cancer cells led to molecular assembly targeting both the plasma membrane and the lysosomes mediated via membrane-bonded enzymes. The dual-targeted-assembly generates a synergistic anticancer effect with amplified cancer spheroid toxicity and enhanced inhibition efficacy on cancer cell migration
- …